Neural networks for variational problems in engineering

In this work a conceptual theory of neural networks (NNs) from the perspective of functional analysis and variational calculus is presented. Within this formulation, the learning problem for the multilayer perceptron lies in terms of finding a function, which is an extremal for some functional. Therefore, a variational formulation for NNs provides a direct method for the solution of variational problems. This proposed method is then applied to distinct types of engineering problems. In particular a shape design, an optimal control and an inverse problem are considered. The selected examples can be solved analytically, which enables a fair comparison with the NN results. Copyright © 2008 John Wiley & Sons, Ltd

An efficient ‘a priori’ model reduction for boundary element models

The Boundary Element Method (BEM) is a discretisation technique for solving partial differential equations, which offers, for certain problems, important advantages over domain techniques. Despite the high CPU time reduction that can be achieved, some 3D problems remain today untreatable because the extremely large number of degrees of freedom—dof—involved in the boundary description. Model reduction seems to be an appealing choice for both, accurate and efficient numerical simulations. However, in the BEM the reduction in the number of degrees of freedom does not imply a significant reduction in the CPU time, because in this technique the more important part of the computing time is spent in the construction of the discrete system of equations. In this way, a reduction also in the number of weighting functions, seems to be a key point to render efficient boundary element simulations

Global search methods for nonlinear optimisation: a new probabilistic-stochastic approach

In this work the problem of overcoming local minima in the solution of nonlinear optimisation problems is addressed. As a first step, the existing nonlinear local and global optimisation methods are reviewed so as to identify their advantages and disadvantages. Then, the major capabilities of a number of successful methods such as genetic, deterministic global optimisation methods and simmulated annealing, are combined to develop an alternative global optimisation approach based on a Stochastic-Probabilistic heuristic. The capabilities, in terms of robustness and efficiency, of this new approach are validated through the solution of a number of nonlinear optimisation problems. A well know evolutionary technique (Differential Evolution) is also considered for the solution of these case studies offering a better insight of the possibilities of the method proposed here.Postprint (published version

An iterative identification procedure for dynamic modeling of biochemical networks

<p>Abstract</p> <p>Background</p> <p>Mathematical models provide abstract representations of the information gained from experimental observations on the structure and function of a particular biological system. Conferring a predictive character on a given mathematical formulation often relies on determining a number of non-measurable parameters that largely condition the model's response. These parameters can be identified by fitting the model to experimental data. However, this fit can only be accomplished when identifiability can be guaranteed.</p> <p>Results</p> <p>We propose a novel iterative identification procedure for detecting and dealing with the lack of identifiability. The procedure involves the following steps: 1) performing a structural identifiability analysis to detect identifiable parameters; 2) globally ranking the parameters to assist in the selection of the most relevant parameters; 3) calibrating the model using global optimization methods; 4) conducting a practical identifiability analysis consisting of two (<it>a priori </it>and <it>a posteriori</it>) phases aimed at evaluating the quality of given experimental designs and of the parameter estimates, respectively and 5) optimal experimental design so as to compute the scheme of experiments that maximizes the quality and quantity of information for fitting the model.</p> <p>Conclusions</p> <p>The presented procedure was used to iteratively identify a mathematical model that describes the NF-<it>κ</it>B regulatory module involving several unknown parameters. We demonstrated the lack of identifiability of the model under typical experimental conditions and computed optimal dynamic experiments that largely improved identifiability properties.</p

Hybrid optimization method with general switching strategy for parameter estimation

This article is available from: http://www.biomedcentral.com/1752-0509/2/26[Background] Modeling and simulation of cellular signaling and metabolic pathways as networks of biochemical reactions yields sets of non-linear ordinary differential equations. These models usually depend on several parameters and initial conditions. If these parameters are unknown, results from simulation studies can be misleading. Such a scenario can be avoided by fitting the model to experimental data before analyzing the system. This involves parameter estimation which is usually performed by minimizing a cost function which quantifies the difference between model predictions and measurements. Mathematically, this is formulated as a non-linear optimization problem which often results to be multi-modal (non-convex), rendering local optimization methods detrimental.[Results] In this work we propose a new hybrid global method, based on the combination of an evolutionary search strategy with a local multiple-shooting approach, which offers a reliable and efficient alternative for the solution of large scale parameter estimation problems.[Conclusion] The presented new hybrid strategy offers two main advantages over previous approaches: First, it is equipped with a switching strategy which allows the systematic determination of the transition from the local to global search. This avoids computationally expensive tests in advance. Second, using multiple-shooting as the local search procedure reduces the multi-modality of the non-linear optimization problem significantly. Because multiple-shooting avoids possible spurious solutions in the vicinity of the global optimum it often outperforms the frequently used initial value approach (single-shooting). Thereby, the use of multiple-shooting yields an enhanced robustness of the hybrid approach.This work was supported by the European Community as part of the FP6 COSBICS Project (STREP FP6-512060), the German Federal Ministry of Education and Research, BMBF-project FRISYS (grant 0313921) and Xunta de Galicia (PGIDIT05PXIC40201PM).Peer reviewe

ASPASIA: A toolkit for evaluating the effects of biological interventions on SBML model behavior

<div><p>A calibrated computational model reflects behaviours that are expected or observed in a complex system, providing a baseline upon which sensitivity analysis techniques can be used to analyse pathways that may impact model responses. However, calibration of a model where a behaviour depends on an intervention introduced after a defined time point is difficult, as model responses may be dependent on the conditions at the time the intervention is applied. We present ASPASIA (Automated Simulation Parameter Alteration and SensItivity Analysis), a cross-platform, open-source Java toolkit that addresses a key deficiency in software tools for understanding the impact an intervention has on system behaviour for models specified in Systems Biology Markup Language (SBML). ASPASIA can generate and modify models using SBML solver output as an initial parameter set, allowing interventions to be applied once a steady state has been reached. Additionally, multiple SBML models can be generated where a subset of parameter values are perturbed using local and global sensitivity analysis techniques, revealing the model’s sensitivity to the intervention. To illustrate the capabilities of ASPASIA, we demonstrate how this tool has generated novel hypotheses regarding the mechanisms by which Th17-cell plasticity may be controlled <i>in vivo</i>. By using ASPASIA in conjunction with an SBML model of Th17-cell polarisation, we predict that promotion of the Th1-associated transcription factor T-bet, rather than inhibition of the Th17-associated transcription factor ROR<i>γ</i>t, is sufficient to drive switching of Th17 cells towards an IFN-<i>γ</i>-producing phenotype. Our approach can be applied to all SBML-encoded models to predict the effect that intervention strategies have on system behaviour. ASPASIA, released under the Artistic License (2.0), can be downloaded from <a href="http://www.york.ac.uk/ycil/software" target="_blank">http://www.york.ac.uk/ycil/software</a>.</p></div

Optimization in computational systems biology

Optimization aims to make a system or design as effective or functional as possible. Mathematical optimization methods are widely used in engineering, economics and science. This commentary is focused on applications of mathematical optimization in computational systems biology. Examples are given where optimization methods are used for topics ranging from model building and optimal experimental design to metabolic engineering and synthetic biology. Finally, several perspectives for future research are outlined

Threshold-Free Population Analysis Identifies Larger DRG Neurons to Respond Stronger to NGF Stimulation

Sensory neurons in dorsal root ganglia (DRG) are highly heterogeneous in terms of cell size, protein expression, and signaling activity. To analyze their heterogeneity, threshold-based methods are commonly used, which often yield highly variable results due to the subjectivity of the individual investigator. In this work, we introduce a threshold-free analysis approach for sparse and highly heterogeneous datasets obtained from cultures of sensory neurons. This approach is based on population estimates and completely free of investigator-set parameters. With a quantitative automated microscope we measured the signaling state of single DRG neurons by immunofluorescently labeling phosphorylated, i.e., activated Erk1/2. The population density of sensory neurons with and without pain-sensitizing nerve growth factor (NGF) treatment was estimated using a kernel density estimator (KDE). By subtraction of both densities and integration of the positive part, a robust estimate for the size of the responsive subpopulations was obtained. To assure sufficiently large datasets, we determined the number of cells required for reliable estimates using a bootstrapping approach. The proposed methods were employed to analyze response kinetics and response amplitude of DRG neurons after NGF stimulation. We thereby determined the portion of NGF responsive cells on a true population basis. The analysis of the dose dependent NGF response unraveled a biphasic behavior, while the study of its time dependence showed a rapid response, which approached a steady state after less than five minutes. Analyzing two parameter correlations, we found that not only the number of responsive small-sized neurons exceeds the number of responsive large-sized neurons—which is commonly reported and could be explained by the excess of small-sized cells—but also the probability that small-sized cells respond to NGF is higher. In contrast, medium-sized and large-sized neurons showed a larger response amplitude in their mean Erk1/2 activity

Identification of neutral biochemical network models from time series data

<p>Abstract</p> <p>Background</p> <p>The major difficulty in modeling biological systems from multivariate time series is the identification of parameter sets that endow a model with dynamical behaviors sufficiently similar to the experimental data. Directly related to this parameter estimation issue is the task of identifying the structure and regulation of ill-characterized systems. Both tasks are simplified if the mathematical model is canonical, <it>i.e</it>., if it is constructed according to strict guidelines.</p> <p>Results</p> <p>In this report, we propose a method for the identification of admissible parameter sets of canonical S-systems from biological time series. The method is based on a Monte Carlo process that is combined with an improved version of our previous parameter optimization algorithm. The method maps the parameter space into the network space, which characterizes the connectivity among components, by creating an ensemble of decoupled S-system models that imitate the dynamical behavior of the time series with sufficient accuracy. The concept of sloppiness is revisited in the context of these S-system models with an exploration not only of different parameter sets that produce similar dynamical behaviors but also different network topologies that yield dynamical similarity.</p> <p>Conclusion</p> <p>The proposed parameter estimation methodology was applied to actual time series data from the glycolytic pathway of the bacterium <it>Lactococcus lactis </it>and led to ensembles of models with different network topologies. In parallel, the parameter optimization algorithm was applied to the same dynamical data upon imposing a pre-specified network topology derived from prior biological knowledge, and the results from both strategies were compared. The results suggest that the proposed method may serve as a powerful exploration tool for testing hypotheses and the design of new experiments.</p